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@ A network of distributed workstations is pro- 
vided for controlling a resource such as a large 
mainframe computer. Server workstations 108. 
110, 112 applying control commands to the 
resource are attached to each resource 102, 
104, 106, preferably through redundant connec- 
tions 114, 118, 116, 120. Remote client consoles 
130 are defined which may be connected to the 
server for control of that resource. Server and 
resource location and primary and fallback con- 
nection paths are maintained by a centralized 
control server 160, 162. Upon client console 
request, the central control server causes the 
server workstation associated with a particular 
resource to establish a control session between 
the server and the dient console. 



PS/2 




-110 ' 



106 



112 



SERVER 



PS/2 



CLIENT 
(CONSO-uE) 



122 



PS/2 



BOA 



Q. 
liJ 



Jouve, 18, rue Saint-Denis, 75001 PARIS 



1 



EP 0 537 903 A2 



2 



The present invention relates to the management 
of distributed hardware resources, for example the 
management of distributed systems that provide re- 
mote operation facilities for computer hardware re- 
sources. 5 

The increasing power and complexity of large 
computer systems, frequently termed "mainframe 
computer systems", has resulted in an increase in the 
complexity of computer system operations. The drive 
for increased workforce productivity, however, has io 
tended to reduce the number of personnel assigned 
to the operations task. The proliferation of large conv 
puter system complexes, such as those used by air- 
line reservations systems, banking centers, and sim- 
ilar computer intensive businesses, has also contrit>- is 
uted to the need for more effective facilities for the 
control of hardware systems resources. 

Large computer systems have traditionally been 
operated from an attached console accessible to the 
computer operators in a computer room. Each conrv 20 
puter system has a dedicated console. Thus, in a 
large complex of, for example, six computers, six op- 
erator consoles require monitoring. Each of these 
computer consoles displays messages In the order 
generated by the computer system. Many of the mes- 25 
sages are Informational, Indicating the status of cer- 
tain operations on the computer systems. Other mes- 
sages provide warnings of current or impending prob- 
lems. Finally, a third class of message requires oper- 
ator response to a request for action, such as mount- 30 
ing a tape, or to correct an error detected in the sys- 
tem. It becomes increasingly difficult for an operator 
to monitor several consoles with several different 
types of messages and be able to effectively respond 
to each one. 35 

One solution to the Increasing flow of messages 
Is to develop an automated operations facility that Is 
able to interpret and classify messages. These types 
of facilities can be constructed to segregate messag- 
es by message type and to present the operator with 40 
those requiring attention. Automated operations fa- 
cilities of this type are typically constructed using a 
workstation computer that can be connected directly 
to the large computer system. The workstation com- 
puter contains the necessary programs for receiving, 45 
analyzing, and responding to certain messages. 

Productivity gains are frequently achieved by 
centralizing operator resources in a single operations 
area. This area may be on a different Floor or in a dif- 
ferent building than the large computers themselves. so 
Centralization requires that remote access and con- 
trol of the hardware resource be provided. However, 
remote access creates a series of problems. 

The first problem is the need to allow access to 
the hardware resource independent of the location of ss 
that resource. Next, recovery from the failure of any 
component in the control system must be possible. In 
other words, control system component failure must 



not cause the failure of control of the larger system. 
Finally, the control system must be flexible allowing 
the addition of controllable resources and individual 
control points without disrupting the ongoing control 
activities. 

The problem of remote operations and manage- 
ment has been addressed in several ways. In U.S. Pa- 
tent Application No. 07/577,967. filed September 4, 
1990, commonly assigned, (EPA 478,942) an auto- 
mated operations system is described which involves 
a controller coupled to the processor with remote 
workstation access for controlling that processor. 
This configuration provides control but limits remote 
access and falls to address the problem of control 
system redundancy and reconfiguration. 

U.S. Patent No. 5,005,122 suggests the use of a 
client server model for network management tasks. 
This system provides for management of a local area 
network (LAN) through the designation of a single 
network management node which directs other 
nodes to perform backup, software distribution, or 
other network management tasks. While this system 
provides a means for managing a network, there is no 
recognition or teaching of the management of a large 
mainframe hardware resource. In particular, there is 
no recognition of the requirement to establish fault tol- 
erant connection facilities between a console client 
and the hardware resource. 

Thus, there remains a technical problem of cre- 
ating a system for remotely controlling a resource 
such as a large computer system in a manner that al- 
lows remote access, failure recovery, and configura- 
tion flexibility. 

Accordingly, the invention provides a system for 
distributed control of one or more hardware resources 
comprising: 

network means for establishing communica- 
tions between a plurality of network elements Includ- 
ing one or more server means for issuing control com- 
mands to said one or more hardware resources, each 
of said server means being attached to one or more 
of said hardware resources, and one or more client 
means for interacting with an operator and for accept- 
ing operator commands to control said one or more 
hardware resources; 

administration means for specifying connec- 
tion paths to each of said hardware resources from at 
least one of said client means through said server 
means for communications therebetween; and 

failure recovery means for re-establishing 
communications between a client means and a server 
means following failure of a previously established 
connection path. 

Thus the system can establish the location of the 
resource to be controlled and create a link between a 
control console and that resource. In addition, the 
system can recognize and recover from the failure of 
any control system component. In a preferred em- 
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bodiment the network means comprises a dual token 
ring network, to provide a fault tolerant capacity in 
case one of the token rings is incapacitated. 

The invention also provides a method for distrib- 
uted control of one or more hardware resources in a 
network including one or more server means for issu- 
ing control commands to said one or more hardware 
resources, each of said server means being attached 
to one or more of said hardware resources, and one 
or more client means for interacting with an operator 
and for accepting operator commands to control said 
one or more hardware resources, said method com- 
prising the steps of: 

specifying network connection paths to each 
of said hardware resources from at least one client 
means through a server means, based on central 
session configuration data stored in a control server; 

establishing communications between the cli- 
ent means and the server means in accordance with 
the specified connection paths; and 

recovering from failure by re-establishing com- 
munications between a client means and a server 
means following failure of a previously established 
connection path. 

Preferably the step of establishing communica- 
tions between a client means and a server means 
comprises the steps of: 

transmitting a session request from said client 
means to the control server specifying a server 
means to be connected; 

accessing session configuration data in said 
control server based upon the client and server con- 
nection requested; 

transmitting configuration data from said con- 
trol server to said server means; and 

establishing a server/client session, said ser- 
ver means initiating said session, based on said fall- 
back configuration data. 

It is further preferred that a failure is recovered 
from by: detecting client control failure in said server 
means; transmitting a fallback request from said ser- 
ver means to said control server; accessing fallback 
configuration information in said control server; 
transmitting fallback information from said control 
server to said server means; and establishing a ser- 
ver/client session, said server means initiating said 
establishment based on said fallback configuration 
data. 

Typically the establishment of a connection 
would result in transmitting a successful session ac- 
knowledgement from said client means to said control 
server and storing said session acknowledgement in 
said control server. 

It is also preferred that a first task operating on a 
server means receives messages from the client 
means, and a second task operating on the server 
means forwards the messages to a hardware re- 
source, a message being passed from the first task 



to the second task using an encrypted security key 
generated by: 

generating a check sum value for the data in 
the message; 

5 copying the check sum value into an address- 

able memory segment of the server means; 

adding a random number to the address of the 
memory segment to create a modified address; 

adding a random number to the check sum val- 
10 ue to create a modified check sum; 

converting the check sum, modified address 
and modif ied check sum into a single string of ASCII 
digits; 

encrypting the string. 

15 Preferably the step of encrypting the string conrv 

prises the steps of exchanging bytes within the single 
string of digits; and encrypting said exchanged bytes 
using an exclusive or function. Once the key has been 
generated it is used in a preferred embodiment by: 

20 passing a message between the first and sec- 

ond tasks, said message including said encrypted se- 
curity key and at least one data field; 

validating the security key in the second task 
by decrypting the key and separating out component 

25 elements for verification against expected values; 

processing said message if said validating is 
successful, and 

terminating with an error message if validation 
is not successful. 

30 It should he noted that the security procedure for 

passing messages between first and second tasks is 
not limited to distributed control systems, but is of po- 
tentially much wider application, and could be used 
for securely transmitting messages between tasks 

35 operating in a computer environment having a proc- 
essor and a memory. 

Thus a computer can be implemented as a sys- 
tem for providing remote control of a resource. The 
system provides a network for maintaining remote 

40 communications between workstations and resourc- 
es to be controlled. It provides redundant control ser- 
vers for actually controlling the resource. Client 
means are connected to the communication network 
and provide an operator interface and operations 

45 control capabilities. Client means further provide an 
ability to accept operator commands and direct them 
to the control server as required. Administration 
means are provided for specifying the locations of re- 
sources and potential connection paths and fallback 

50 paths for those resources. Finally, failure recovery 
means are provided for detecting and correcting con- 
trol system failure by re-establishing dient-to-re- 
source connections. 

A computer implemented system is therefore 

55 used for distributed control of hardware resources. 
The present system provides a renrKDtely distributed 
network that allows communication between a series 
of network components that include servers for con- 
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trolling a hardware resource and clients for interacting 
with an operator. Means are also provided for admin- 
istrating a control network such that connections are 
established between the elements as required and 
that any failure of connections is detected and cor- 
rected through the use of redundant network compo- 
nents, A record of each dynamic connection is main- 
tained by the central control server. Upon failure of a 
client console, the server accesses central server 
fallback information and establishes a fallback con- 
nection with an operational client console. Failure of 
a server workstation causes the resource to establish 
a link with a backup server which is then able to re- 
cover alt sessions with client consoles by accessing 
the dynamic configuration database of the central 
controller. Configuration data in the control system 
may be updated dynamically taking effect when the 
next request for establishment of a session is made. 

An embodiment of the invention will now be de- 
scribed by way of example with reference to the fol- 
lowing drawings: 

Fig. 1 is a block diagram illustrating a system ac- 
cording to the present invention; 
Fig. 2 is a block diagram illustrating the steps of 
establishing an initial client console-to-server 
session; 

Fig. 3 is a flowchart illustrating the process steps 
for establishing an initial client/sever session; 
Fig. 4 is a block diagram illustrating the step of re- 
questing creation of a session between a client 
and a host resource; 

Fig. 5 is a block diagram illustrating the actions 
occurring upon the failure of a client console 
workstation; 

Fig. 6 is a flowchart illustrating the process steps 
of recovering from a client failure; 
Fig. 7 is a block diagram illustrating recovery from 
a server failure; 

Fig. 8 is a flowchart illustrating the process steps 
for recovering from a server failure; 
Fig. 9 is a block diagram illustrating the adminis- 
trative update of configuration data; and 
Fig. 10 is a flowchart depicting interprocess se- 
curity key generation. 

As shown in Figure 1, a centralised control proc- 
ess for a distributed resource controller manages 
hardware resources, eg to control the operation of a 
mainframe computer system such as the IBM 3090 
computer system (IBM and 3090 are trademarks of 
the IBM Corporation). The system can also be used 
to control other resources which require operator in- 
teraction including resources such as a production 
assembly line, chemical processing facility, or build- 
ing environmental system. The structure shown in 
Fig. 1 has resources to be controlled, identified gen- 
erally at 102, 104, and 106. It will be recognized that 
this system is applicable to any number of resources 
and the example of three is shown for illustrative pur- 



poses only. 

Each resource is connected to at least one server 
workstation. Server workstations are shown at 108, 
110, and 112, These workstations are an intelligent 

5 device such as an IBM Personal System/2 computer 
system (PS/2) (Personal System/2 and PS/2 are 
trademarks of the IBM Corporation). The servers are 
connected to hardware resources by communication 
lines 114. 116, 118. 120 and 122. In the preferred env 

10 bodiment, a resource typically is connected to more 
than one server to provide a fallback control path. For 
example, resource 102 is connected via communica- 
tion path 114 to server 108 and by communication 
path 116 to server 110. 

15 A client workstation 130 (such as an IBM PS/2 

computer system) is provided as a console for oper- 
ator use. The client workstation manages the opera- 
tor interaction including the presentation of messag- 
es to the operator and the generation or acceptance 

20 of commands for controlling the hardware resource. 
The separation of client and server functions allows 
a balancing of workload between the devices. It also 
allows a single client console to manage a number of 
hardware resources. In the example shown in Fig. 1, 

25 client console 130 can manage resources 102, 104, 
and 106. 

The client console communicates with the ser- 
vers by means of a local area network (LAN) shown 
generally at 150. This network can be anyone of sev- 

30 eral known networks, such as the IBM Token Ring 
LAN. an Ethernet LAN. or others. In a preferred em- 
bodiment, the control scheme is implemented using 
dual token ring LANS which provide for redundancy in 
case of the failure by either individual LAN. 

35 A central control facility containing control and ad- 

ministrative applications (CAA) is provided to control 
the operation of the control system. This control ser- 
ver 160 is a workstation which has been configured 
to support and control the overall operation of the re- 

40 source control system. The control server provides 
centralized control to ensure that console access ex- 
ists for each resource to be controlled and that the cli- 
ent workstations know how to access each resource 
and how to recover from component failure. Control 

45 server 160 manages the network based upon config- 
uration data stored in a database 162. This data can 
be stored in a variety of known forms such as files on 
the server or data under the control of the database 
management system. 

50 The configuration data consists of two major 

types of data. First, static configuration data contains 
a description of each hardware resource location to 
be controlled. It also contains the parameters neces- 
sary for accessing and controlling that resource and 

55 identifies primary and fallback access paths to that 
resource. The second type of data is dynamic config- 
uration data describing the current configuration for 
controlling each resource. This data is maintained to 
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assure that eacti resource is under the control of a 
console and for use by fallback processing routines to 
re-establish communications in the case of a failure. 

The operation of the system to establish a re- 
source control environment will be described with ref- 
erence to Fig. 2. A resource control environment in- 
cludes a console for operator interaction, control rou- 
tines and communications processes for hardware re- 
source management. Fig. 3 is a flowchart presenting 
the process steps required to establish the control en- 
vironment 

The control environment for a particular hard- 
ware resource is initiated by an operator requesting 
that a console session be established between a par- 
ticular client console and a resource. The client con- 
sole is typically distributed remotely from the hard- 
ware resource and a network session must be estat>- 
lished between the client console and the resource. 
The operator requests an access to a resource by se- 
lecting that resource from a list of resources present- 
ed in a user interface. Fig. 4 illustrates the network 
accesses required to establish a console session with 
a particular resource. Screen image 402 is an exam- 
ple of a visual Interface to the operator presenting a 
list of resources that may be selected for console con- 
nection. The operator uses a cursor control device, 
such as a mouse, to select a resource from the menu 
provided. The requestor program 404 operating in 
workstation 130 uses the services of a workstation 
information manager 408 to determine the location of 
the control server CAA 160. The location of the con- 
trol database is specified in a data segment 410 and 
is used by the workstation for access over token ring 
LAN 150 to the control server 160. 

Next (step 302 in Fig. 3) the control server 160 
accesses configuration database 162 to determine 
the location of the hardware resource to be accessed 
by the console, in the example in Fig. 2, resource 102' 
is to be accessed. The configuration data 162 will de- 
termine that resource 102' is controlled by server 
108' and will initiate a session startup 304 by sending 
a message to server 108', which is responsible for es- 
tablishing a control session (step 306) with the client 
console 130. Once the session is successfully estab- 
lished over the network, the client console 1 30 sends 
(step 308) a successful session startup acknowledge- 
ment to the control server 1 60. The control server will 
then record 310 the session startup information in 
the dynamic data portion of configuration database 
162. 

Fig. 5 illustrates the message flow for recovering 
from a client console failure. Fig. 6 is a flowchart de- 
scribing that process. In step 610, the server 108' de- 
tects the failure of the network session between it and 
the client console 130. Server 108' sends a fallback 
request to control server 160 (step 612). The control 
server accesses the configuration database 162 to 
determine configuration fallback data. Fallback infor- 



mation is sent 616 to server 108' that then estab- 
lishes a new session 618 with the fallback client con- 
sole, in this case, console 132. Client console 132 
sends an acknowledgement 620 to the control server 
5 160 when the session is successfully established. 
Control server 160 records the new session informa- 
tion 622 in the dynamic portion of configuration da- 
tabase 162. 

Fig. 7 illustrates the process for recovering from 

10 the failure of a resource server and Fig. 8 lists the 
process steps for that recovery. The failure of a ser- 
ver must be detectable by the resource under control 
102' which then must have the ability to switch to a 
backup server. Upon detection of server failure 810. 

15 resource 102' initiates control switch to backup server 
110'. Server 110' recognizes the resource action and 
issues 414 a fallback request to the control server 
160. Control server 160 accesses the configuration 
database to determine the client console sessions 

20 impacted by the failure of server 108'. The dynamic 
configuration data indicates all connected sessions 
and can be used by the fallback server 110' to re-es- 
tablish those connections. The fallback information is 
sent 118 to server 110'. Server 110' establishes new 

25 connections with each client console previously con- 
nected to server 108', for example, client console 
130. Upon successful session establishment client 
console 130 sends an acknowledgement of the new 
session 822 to the control server 160. The control 

30 server records the new session information in the dy- 
namic portion of the configuration database 162 as 
step 124. 

Interprocess security can be used to enhance the 
reliability of server tasks. The control server 160 and 

35 hardware resource servers (e.g. 108') operate using 
"frontend" tasks for network communication and 
"backend" tasks for configuration control and re- 
source control. The division of processing tasks into 
frontend and backend tasks raises the issue of secur- 

40 ity in the communications between those frontend 
and backend tasks. Because the backend tasks inter- 
act with computer system resources and thereby 
have a major impact upon computer system opera- 
tion, security of messages and data transmitted to the 

45 backend tasks is important The backend tasks must 
be able to ensure that the messages it receives ori- 
ginated from an authorized frontend task and not an- 
other unauthorized program attempting to manipu- 
late the system resources. Second, the backend task 

50 must be able to detect possible modifications of data 
during the transmission process from an authorized 
frontend to a backend. Finally, the backend must be 
able to detect the situation where authorized frontend 
information is captured, modified, and later transmit- 

55 ted to the backend. The solution to this problem is to 
develop a security key to be combined with the data 
sent between the front and backends. This key must 
be such that interception and modification of an au- 
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thorized message is detected and that it be difficult 
to decompose the key to reverse engineer the secur- 
ity algorithm. 

A security key meeting these requirements is 
constructed on the basis of three values. First, the 
address of a shared memory segment sharable be- 
tween frontend and backend tasks. Second, a stan- 
dard check sum of the data. Finally, a random nunrv 
ber. The combination of these three components pro- 
vides a secure key meeting the objectives of the se- 
curity system. First, the address of a shared memory 
segment indicates the task sending the data is autho- 
rized to operate on the computer system and have ac- 
cess to the shared memory. The check sum of the 
data ensures that the data received by the backend 
task has not been modified enroute or captured and 
modified. Finally, the random number introduces a 
degree of variability and randomness into the key. 

The generated key is created according to the 
process shown in Fig. 10. This operates as follows. 

The checksum is first copied into a shared mem- 
ory segment of known address. The random number 
is added to the adress of the shared memory seg- 
ment. The random number is added to the check sum 
value. All three numbers are converted into ASCII dig- 
its. The results of the conversion are concatenated 
into a single string of digits. The bytes in the concat- 
enated string are exchanged according to a predeter- 
mined pattern. The resulting character string is en- 
crypted using a logical "exclusive or" operation on a 
character- by-character basis with a known static val- 
ue. 

The security key for the message is sent with the 
data to the backend task. The backend task validates 
the key by reversing the above construction process. 

This method of constructing the key is unique be- 
cause it combines elements derived from the calling 
environment (the address of the shared memory seg- 
ment), along with elements derived from the actual 
data sent (the check sum value), and a random factor 
to help conceal the security method and protect 
against attempts to reverse engineer the key struc- 
ture algorithm. 

Fig. 9 illustrates the update network configura- 
tion which allows for dynamically updating the config- 
uration information in configuration database 162. An 
administrative program operates in a client server, for 
example, client server 134. (The administrative pro- 
gram could also operate on one of the console client 
servers, e.g.. 130 or 132). The administrative pro- 
gram collects the information necessary to update 
the configuration data to add a resource, or -to 
change resource access paths or fallback paths and 
transmits it for update over token ring l^N 1 50. In the 
preferred embodiment, the database update is ac- 
complished using the remote data services facility of 
the Operating System/2 (OS/2) operating system 
(OS/2 is a trademark of the IBM Corporation). The up- 



date is accomplished by modifying configuration da- 
tabase 162. The modified configuration information 
will take effect whenever control server 160 is re- 
quired to establish a new session between client and 
5 server workstations. Existing sessions will continue 
to process as originally set up until manually termin- 
ated or until a failure is detected, in which case the 
above-described failure recovery process will be em- 
ployed using the updated configuration information. 

10 

Claims 

1. A system for distributed control of one or more 
15 hardware resources (102. 104, 106) comprising: 

network means (150) for establishing 
communications between a plurality of network 
elements including one or more server means 
(108, 110. 112) for issuing control commands to 

20 said one or more hardware resources, each of 

said server means being attached to one or more 
of said hardware resources, and one or more cli- 
ent means (130) for interacting with an operator 
and for accepting operator commands to control 

25 said one or more hardware resources; 

administration means (160, 162) for spec- 
ifying connection paths to each of said hardware 
resources from at least one of said client means 
through said server means for communications 

30 therebetween; and 

failure recovery means for re-establishing 
communications between a client means and a 
server means following failure of a previously es- 
tablished connection path. 

35 

2. The system of Claim 1 wherein said network 
means comprises a dual token ring network, 

3. A method for distributed control of one or more 
40 hardware resources (1 02, 1 04, 1 06) in a network 

including one or more server means (108, 110. 
112) for issuing control commands to said one or 
more hardware resources, each of said server 
means being attached to one or more of said 

45 hardware resources, and one or more client 

means (130) for interacting with an operator and 
for accepting operator commands to control said x 
one or more hardware resources, said method 
comprising the steps of: 

50 specifying network connection paths to 

each of said hardware resources from at least 
one client means through a server means, based 
on central session configuration data stored in a 
control server; 

55 establishing communications between the 

client means and the server means in accor- 
dance with the specified connection paths; and 
recovering from failure by re-establishing 
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communications between a client means and a 
server means following failure of a previously es- 
tablished connection path. 

The method of daim 3, wherein the step of estate- 5 
lishing communications between a client means 
and a server means comprises the steps of: 

transmitting a session request from said 
client means to the control server (160) specify- 
ing a server means to be connected; io 

accessing session configuration data 
(162) in said control server based upon the client 
and server connection requested; 

transmitting configuration data from said 
control server to said server means; and is 

establishing a server/client session, said 
server means initiating said session, based on 
said fallback configuration data 

The method of claim 4, further including recover- 20 
ing from a failure by: 

detecting dient control failure in said ser- 
ver means; 

transmitting a fallback request from said 
server means to said control server; 25 

accessing fallback configuration informa- 
tion in said control server, 

transmitting fallback information from said 
control server to said server means; and 

establishing a server/dient session, said 30 
server means initiating said establishment based 
on sard fallback configuration data. 

The method of daim 4 or 5 further comprising the 
steps of: 35 

transmitting a successful session ac- 
knowledgement from said dient means to said 
control server; and 

storing said session acknoledgement in 
said control server. 40 

The method of any of claims 3 to 6» wherein a first 
task operating on a server means receives mes- 
sages from the dient means, and a second task 
operating on the server means forwards the 45 
messages to a hardware resource, a message 
being passed from the first task to the second 
task using an encrypted security key generated 
by: 

generating a check sum value for the data so 
in the message; 

copying the check sum value into an ad- 
dressable memory segment of the server means; 

adding a random number to the address of 
the memory segment to create a modified ad- ss 
dress; 

adding a random number to the check sum 
value to create a modified check sum; 



converting the check sum, modified ad- 
dress and modified check sum into a single string 
of ASCII digits; 

encrypting the string. 

8. The method of Oaim 7 wherein the step of en- 
crypting the string comprises the steps of: 

exchanging bytes within the single string 
of digits; and 

encrypting said exchanged bytes using an 
exdusive or function. 

9. The method of claim 7 or 8, further comprising 
the steps of: 

passing a message between the first and 
second tasks, said message induding said en- 
crypted security key and at least one data field; 

validating the security key in the second 
task by decrypting the key and separating out 
component elements for verification against ex- 
pected values; 

processing said message if said validating 
is successful, and 

terminating with an error message if vali- 
dation is not successful. 
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